Online Learning and Mistake Bounds for Finite Hypothesis Classes

In this lecture we describe a different model of learning which is called online learning. Online learning takes place in a sequence of consecutive rounds. To demonstrate the online learning model, consider again the papaya tasting problem. On each online round, the learner first receives an instance (the learner buys a papaya and knows its shape and color, which form the instance). Then, the learner is required to predict a label (is the papaya tasty?). At the end of the round, the learner gets the correct label (he tastes the papaya and then knows if it’s tasty or not). Finally, the learner uses this information to improve his future predictions. Previously, we used the batch learning model in which we first use a batch of training examples to learn a hypothesis and only when learning is completed the learned hypothesis is tested. In our papayas learning problem, we should first buy bunch of papayas and taste them all. Then, we use all of this information to learn a prediction rule that determines the taste of new papayas. In contrast, in online learning there is no separation between a training phase and a test phase. The same sequence of examples is used both for training and testing and the distinguish between train and test is through time. In our papaya problem, each time we buy a papaya, it is first considered a test example since we should predict if it’s going to taste good. But, after we take a byte from the papaya, we know the true label, and the same papaya becomes a training example that can help us improve our prediction mechanism for future papayas. The goal of the online learner is simply to make few prediction mistakes. By now, the reader should know that there are no free lunches – we must have some prior knowledge on the problem in order to be able to make accurate predictions. As in previous lectures, we encode our prior knowledge on the problem using some representation of the instances (e.g. shape and color) and by assuming that there is a class of hypotheses, H = {h : X → Y}, and on each online round the learner uses a hypothesis from H to make his prediction. To simplify our presentation, we start the lecture by describing online learning algorithms for the case of a finite hypothesis class.